The combination of transformers and masked image modeling (MIM) pre-training framework has shown great potential in various vision tasks. However, the pre-training computational budget is too heavy and withholds the MIM from becoming a practical training paradigm. This paper presents FastMIM, a simple and generic framework for expediting masked image modeling with the following two steps: (i) pre-training vision backbones with low-resolution input images; and (ii) reconstructing Histograms of Oriented Gradients (HOG) feature instead of original RGB values of the input images. In addition, we propose FastMIM-P to progressively enlarge the input resolution during pre-training stage to further enhance the transfer results of models with high capacity. We point out that: (i) a wide range of input resolutions in pre-training phase can lead to similar performances in fine-tuning phase and downstream tasks such as detection and segmentation; (ii) the shallow layers of encoder are more important during pre-training and discarding last several layers can speed up the training stage with no harm to fine-tuning performance; (iii) the decoder should match the size of selected network; and (iv) HOG is more stable than RGB values when resolution transfers;. Equipped with FastMIM, all kinds of vision backbones can be pre-trained in an efficient way. For example, we can achieve 83.8%/84.1% top-1 accuracy on ImageNet-1K with ViT-B/Swin-B as backbones. Compared to previous relevant approaches, we can achieve comparable or better top-1 accuracy while accelerate the training procedure by $\sim$5$\times$. Code can be found in https://github.com/ggjy/FastMIM.pytorch.

网络架构在基于深度学习的计算机视觉系统中起关键作用。广泛使用的卷积神经网络和变压器将图像视为网格或序列结构，该网格或序列结构并非灵活以捕获不规则和复杂的对象。在本文中，我们建议将图像表示为图形结构，并引入新的视觉GNN（VIG）体系结构，以提取视觉任务的图形级特征。我们首先将图像拆分为许多被视为节点的补丁，然后通过连接最近的邻居来构造图形。根据图像的图表表示，我们构建了VIG模型以在所有节点之间转换和交换信息。 VIG由两个基本模块组成：用于汇总和更新图形信息的图形卷积的图形模块，以及带有两个线性层的FFN模块用于节点特征转换。 VIG的各向同性和金字塔体系结构均具有不同的型号。关于图像识别和对象检测任务的广泛实验证明了我们的VIG架构的优势。我们希望GNN关于一般视觉任务的开创性研究将为未来的研究提供有用的灵感和经验。 pytorch代码可在https://github.com/huawei-noah/effficity-ai-backbones上获得，Mindspore代码可在https://gitee.com/mindspore/models上获得。

变压器网络对计算机视觉任务取得了很大的进步。变压器 - 变压器（TNT）架构利用内部变压器和外部变压器提取本地和全局表示。在这项工作中，我们通过引入两个先进的设计：1）金字塔架构和2）卷积阀。通过建立分层表示，新的“金字塔”显着改善了原始TNT。Pyramidtnt比以前的最先进的视觉变压器（如Swin Transformer）实现更好的表演。我们希望这一新基线能够有助于视觉变压器的进一步研究和应用。代码将在https://github.com/huawei-noah/cv-backbones/tree/master/tnt_pytorch获得。

与传统的卷积神经网络（CNN）和视觉变压器不同，多层默认（MLP）是一种新的视觉模型，具有极其简单的架构，其仅由完全连接的层堆叠。 Vision MLP的输入图像通常被分成多个令牌（补丁），而现有的MLP模型直接用固定权重聚合它们，忽略来自不同图像的令牌的变化语义信息。为了动态聚合令牌，我们建议将每个令牌代表为具有两个部分，幅度和相位的波函数。幅度是原始特征，并且相位项是根据输入图像的语义内容改变的复值。介绍相位项可以动态调制MLP中令牌和固定权重之间的关系。基于波浪状令牌表示，我们建立了一种用于视觉任务的新型波-MLP架构。广泛的实验表明，所提出的波-MLP优于各种视觉任务的最先进的MLP架构，例如图像分类，对象检测和语义分割。

先前的视觉MLP，如MLP-MILER和RESMLP接受线性扁平的图像贴片作为输入，使其对不同的输入大小和难以捕获空间信息。这种方法隐瞒了MLP与基于变压器的对应物相比，并防止它们成为计算机视觉的一般骨干。本文介绍了Hire-MLP，通过\ TextBF {Hi} reachical \ TextBF {Re}排列，这是一个简单而竞争的愿景MLP架构，其中包含两个重排级别。具体地，提出内部区域重新排列以捕获空间区域内的局部信息，并且提出横区域重新排列以使不同区域之间的信息通信能够通过沿空间方向循环地转换所有令牌来实现不同区域之间的信息通信。广泛的实验证明了Hire-MLP作为各种视觉任务的多功能骨干的有效性。特别是，Hire-MLP在图像分类，对象检测和语义分割任务上实现竞争结果，例如，在Imagenet上的83.8％的前1个精度，51.7％盒AP和Coco Val2017上的44.8％掩模AP和Ade20k上的49.9％Miou ，超越以前的基于变压器和基于MLP的型号，具有更好的折衷以获得准确性和吞吐量。代码可在https://github.com/ggjy/hire-wave-mlp.pytorch获得。

视觉变压器由于能够捕获图像中的长期依赖性的能力而成功地应用于图像识别任务。但是，变压器与现有卷积神经网络（CNN）之间的性能和计算成本仍然存在差距。在本文中，我们旨在解决此问题，并开发一个网络，该网络不仅可以超越规范变压器，而且可以超越高性能卷积模型。我们通过利用变压器来捕获长期依赖性和CNN来建模本地特征，从而提出了一个新的基于变压器的混合网络。此外，我们将其扩展为获得一个称为CMT的模型家族，比以前的基于卷积和基于变压器的模型获得了更好的准确性和效率。特别是，我们的CMT-S在ImageNet上获得了83.5％的TOP-1精度，而在拖鞋上的拖曳率分别比现有的DEIT和EficitiveNet小14倍和2倍。拟议的CMT-S还可以很好地概括CIFAR10（99.2％），CIFAR100（91.7％），花（98.7％）以及其他具有挑战性的视觉数据集，例如可可（44.3％地图），计算成本较小。

二进制神经网络（BNNS）将原始的全精度权重和激活为1位，带有符号功能。由于传统符号函数的梯度几乎归零，因此不能用于反向传播，因此已经提出了几次尝试来通过使用近似梯度来缓解优化难度。然而，这些近似损坏了事实梯度的主要方向。为此，我们建议使用用于训练BNN的正弦函数的组合来估计傅立叶频域中的符号功能的梯度，即频域近似（FDA）。该提出的方法不会影响占据大部分整体能量的原始符号功能的低频信息，并且将忽略高频系数以避免巨大的计算开销。此外，我们将噪声适配模块嵌入到训练阶段以补偿近似误差。关于多个基准数据集和神经架构的实验说明了使用我们的方法学习的二进制网络实现了最先进的准确性。代码将在\ texit {https://gitee.com/mindspore/models/tree/master/research/cv/fda-bnn}上获得。

As one of the most important psychic stress reactions, micro-expressions (MEs), are spontaneous and transient facial expressions that can reveal the genuine emotions of human beings. Thus, recognizing MEs (MER) automatically is becoming increasingly crucial in the field of affective computing, and provides essential technical support in lie detection, psychological analysis and other areas. However, the lack of abundant ME data seriously restricts the development of cutting-edge data-driven MER models. Despite the recent efforts of several spontaneous ME datasets to alleviate this problem, it is still a tiny amount of work. To solve the problem of ME data hunger, we construct a dynamic spontaneous ME dataset with the largest current ME data scale, called DFME (Dynamic Facial Micro-expressions), which includes 7,526 well-labeled ME videos induced by 671 participants and annotated by more than 20 annotators throughout three years. Afterwards, we adopt four classical spatiotemporal feature learning models on DFME to perform MER experiments to objectively verify the validity of DFME dataset. In addition, we explore different solutions to the class imbalance and key-frame sequence sampling problems in dynamic MER respectively on DFME, so as to provide a valuable reference for future research. The comprehensive experimental results show that our DFME dataset can facilitate the research of automatic MER, and provide a new benchmark for MER. DFME will be published via https://mea-lab-421.github.io.

Reading comprehension of legal text can be a particularly challenging task due to the length and complexity of legal clauses and a shortage of expert-annotated datasets. To address this challenge, we introduce the Merger Agreement Understanding Dataset (MAUD), an expert-annotated reading comprehension dataset based on the American Bar Association's 2021 Public Target Deal Points Study, with over 39,000 examples and over 47,000 total annotations. Our fine-tuned Transformer baselines show promising results, with models performing well above random on most questions. However, on a large subset of questions, there is still room for significant improvement. As the only expert-annotated merger agreement dataset, MAUD is valuable as a benchmark for both the legal profession and the NLP community.

An increasing number of public datasets have shown a marked clinical impact on assessing anatomical structures. However, each of the datasets is small, partially labeled, and rarely investigates severe tumor subjects. Moreover, current models are limited to segmenting specific organs/tumors, which can not be extended to novel domains and classes. To tackle these limitations, we introduce embedding learned from Contrastive Language-Image Pre-training (CLIP) to segmentation models, dubbed the CLIP-Driven Universal Model. The Universal Model can better segment 25 organs and 6 types of tumors by exploiting the semantic relationship between abdominal structures. The model is developed from an assembly of 14 datasets with 3,410 CT scans and evaluated on 6,162 external CT scans from 3 datasets. We rank first on the public leaderboard of the Medical Segmentation Decathlon (MSD) and achieve the state-of-the-art results on Beyond The Cranial Vault (BTCV). Compared with dataset-specific models, the Universal Model is computationally more efficient (6x faster), generalizes better to CT scans from varying sites, and shows stronger transfer learning performance on novel tasks. The design of CLIP embedding enables the Universal Model to be easily extended to new classes without catastrophically forgetting the previously learned classes.